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INTRODUCTION 


\. 

--  Over  the  past  three  decades  the  business  of  vulnerability 
analysis  has  been  conducted  more  as  art  than  as  science.  For  a 
number  of  reasons,  there  has  been  recent  movement  away  from  art 

and  towards  science^.. - Qr^  reason  is  internal  to  the  Ballistic 

Research  Laboratory  If  a  substantial  work  force  of  scien¬ 
tists  and  engineers  spends  several  decades  doing  vulnerability 
analysis  it  would  be  both  surprising  and  disappointing  if 
improved  clarity  on  the  foundations  of  the  subject  were  not 
achieved.  A  second  reason  is  the  declining  cachet  associated 
with  computer  models.  Several  decades  ago  it  was  virtually  the 
case  that  if  a  (^antitative  result  issued  from  a  computer  model, 
it  was  sufficiently  warranted  as  correct.  The  crude  extrapola¬ 
tions  which  then  passed  as  vulnerability  models  benefited  from 
that  halo  effect.  Today,  people  are  understandably  more  skepti¬ 
cal  about  models.  Questions  are  asked  and  answers  expected  about 
accuracy  and  validation.  The  pressure  of  such  questioning  has 
forced  vulnerability  analysts  into  seeking  greater  scientific 
respectability  for  the  fruits  of  their  labors.  A  third  cause  of 
the  scientific  turn  is  the  recent  explosion  of  computer  technol¬ 
ogy.  Vulnerability  phenomena  are  complex;  realistic  simulation 
was  out  of  the  question  with  the  tools  available  to  early  vulner¬ 
ability  analysts.  ^ C _ 

Intersection  of  the  three  trends  mentioned  has  caused  the 
progress  of  vulnerability  analysis  towards  science  to  proceed  at 
an  uneven  rate.  Accurate  computerized  target  descriptions  seem 
clearly  to  be  a  necessary  condition  on  scientifically  adequate 
vulnerability  modeling  and  a  set  of  appropriate  tools  is  in 
hand.  The  stochastic  nature  of  vulnerability  phenomena  has  been 
more  explicitly  recognized,  and  appropriate  modeling  tools 
developed. 


1.  Ballistic  Research  Laboratory  CAD  Package,  Release  3.0.  (1 

Oct  88) ,  "A  Solid  Modeling  Systems  and  Ray-Tracing  Bench¬ 
mark  Distribution  Package,”  SECAD/VLD  Computing  Consortium, 
The  US  Army  Ballistic  Research  Laboratory,  Aberdeen  Proving 
Ground,  MD  21005-5066 

2.  A.  Ozolins,  "Stochastic  High  Resolution  Vulnerability  Simu¬ 
lation  for  Live  Fire  Programs,"  in  The  Proceedings  of  the 
Tenth  Annual  Symposium  on  Survivability  and  Vulnerability 
of  the  American  Defense  Preparedness  Association  (ADPA) , 
10-12  May  1988. 
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Vulnerability  measures  of  effectiveness  have  been  more  clearly 
defined  than  was  traditionally  the  case.  However,  one  area 
where  less  progress  has  been  made  has  to  do  with  issues  of  vul¬ 
nerability  model  validation  and  accuracy. 

Most  of  the  personnel  resources  of  the  Ballistic  Research 
Laboratory's  Vulnerability/  Lethality  Division  (VLD)  are  devoted 
to  predicting  the  vulnerability  of  targets  to  conventional  threat 
munitions  or  to  predicting  the  lethality  of  conventional  muni¬ 
tions  when  fired  against  various  targets.  No  doubt,  some  would 
prefer  to  say  that  VLD  estimates  vulnerability  rather  than 
predicts  it.  This  strikes  me  as  a  euphemistic  way  of  avoiding 
the  question  of  how  good  our  predictions  are. 

In  this  paper  I  will  discuss  the  idea  of  an  accurate  vulnera¬ 
bility  prediction  and  related  notions.  Let  me  begin  by  beating 
the  surrounding  bushes  to  clarify  what  the  paper  is  not  about. 
There  is  no  attempt  to  answer  the  question  whether  current  vul¬ 
nerability  models  are  accurate,  or  even  to  lay  out  the  details  of 
a  practical  program  for  determining  whether  they  are  accurate. 
Nor  do  I  discuss  the  empirical  data  voids  which  clearly  contri¬ 
bute  to  inaccurate  vulnerability  predictions.  I  do  hope  to  pro¬ 
vide  a  consistent  framework  within  which  such  issues  can  be 
coherently  debated.  This  would  itself  constitute  progress. 

There  is  a  sense  in  which  to  speak  accurately  is  to  speak 
precisely,  but  that  is  not  the  operative  sense  here.  The  proba¬ 
bility  of  throwing  snake  eyes  with  fair  dice  is 
.0277777777777778.  The  assertion  meets  reasonable  scientific 
standards  for  both  precision  and  accuracy,  though  depending  on 
the  intended  application  one  might  argue  that  it  is  needlessly 
precise. 

Suppose  you  are  now  asked  to  interpret  the  stated  probability 
as  the  Probability  of  kill  (Pk)  of  the  M107  HE  round  versus  the 
Super  Soviet  Tank  (SST) .  Arguments  about  precision  will  be  the 
same  as  for  the  dice  throw.  With  respect  to  accuracy,  though, 
virtually  everyone  will  agree  that  our  Pk  is  not  so  accurate  as 
the  stated  precision  reflects.  It  will  be  correctly  argued  that 
our  models  are  simply  not  that  good  and  that  the  required  inputs 
are  not  sufficiently  well-known.  Heads  will  nod  knowingly  that 


3.  J.M.  Abell,  L.K.  Roach,  and  M.  Starks,  "Degraded  States 
Vulnerability  Analysis,"  BRL  Report  in  press. 
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our  Pk  is  not  sufficiently  accurate  to  justify  sixteen  place  pre¬ 
cision.  Those  same  heads  will  also  nod  approvingly  if  we  say 
that  the  Pk=.03.  In  truth,  however,  we  cannot,  or  at  least  have 
not  clearly  stated  the  grounds  according  to  which  our  answer  is 
accurate  to  two  digit  precision.  Let  us  begin  with  an  overview 
of  the  vulnerability  analysis  process. 

2.  Conceptual  Spaces  for  Vulnerability  Analysis 

A  paper  by  Deitz  and  Ozolins  usefully  divides  the  vulnerabilj 
ity  assessment/analysis  process  into  four  distinct  spaces. 
These  conceptual  spaces  are  equally  applicable  to  the  conduct  and 
analysis  of  Live-Fire  Testing  or  to  a  wholly  simulated  analysis. 
Points  in  a  lower  number  space  are  mapped  to  higher  spaces  by 
experimental  processes  and/or  definitional  and  mathematical 
transforms . 

As  illustrated  in  Figure  1,  Space  1  contains  the  details  con¬ 
cerning  interaction  of  a  specific  munition  against  a  specific 
target.  With  respect  to  a  munition,  this  includes  the  mass, 
velocity,  shape,  orientation,  etc.  In  terms  of  the  target,  the 
specifics  include  all  of  the  three-dimensional  geometry  (includ¬ 
ing  armor  packages  and  interior  components) ,  material  properties, 
interdependency  of  system  functioning,  etc.,  and  the  munition 
impact  location. 

Whether  a  real  bullet  is  fired  against  a  target  in  a  live- 
fire  experiment  or  a  computer-based  simulation  is  performed  to 
that  end,  damage  to  the  target  can  occur  as  a  result  of  the 
interaction.  In  the  case  of  an  undermatching  munition,  it  may  be 
that  no  damage  occurs.  In  any  case,  the  state  of  the  target 
after  the  interaction  is  defined  in  terms  of  the  vehicle  critical 
components.  A  critical  component  is  any  component,  the  loss  of 
which  would  result  in  the  reduction  in  a  mobility  or  firepower 
capability  of  the  vehicle.  Past  and  current  practice  in  vulnera¬ 
bility  assessment  is  to  describe  individual  components  in  crisp 
binary  states,  i.e.  killed  or  not  killed.  At  the  component  level 
no  partial  functioning  is  allowed.  (See  Appendix  A)  Following  a 
shot,  the  damage  state  of  the  vehicle  is  defined  as  the  full 
accounting  of  all  vehicle  critical  components.  Each  point  within 
Space  1  represents  one  of  a  large  number  of  possible 
bullet/target  interactions.  As  noted  above  each  specific 
bullet/target  state  is  characterized  by  literally  hundreds  of 


4.  P.H.  Deitz  and  A.  Ozolins,  Computer  Simulations  of  the 
Abrams  Live-Fire  Field  Testing,  in  the  Proceedings  of  AORS 
XXVIII,  12-13  October  1988.  (The  next  four  paragraphs 
closely  follow  this  paper) . 
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Warhead/Target  Interaction 


2  Component  Damage  State(s) 


3  Measures  of  Performance  (MOPs) 
[  Loss  of  Automotive/  Firepower 
Capabilities  ] 


4  Measures  of  Effectiveness  (MOEs) 
[  Reduction  in  Battlefield  Utility, 
"PKs",  or  "Losses-of-Function"] 


Figure  1.  Conceptual  Spaces  for  Vulnerability  Modeling 
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thousands  of  numbers  representing  the  state  of  the  system 
geometry,  material  constituencies,  component  interconnectivities, 
warhead  penetration  performance  parameters,  etc.  The  many  points 
in  Space  2  imply  a  large  number  of  possible  outcomes  that  may 
occur  following  a  bullet/target  interaction.  If  a  target  is  con¬ 
structed  of  n  critical  components,  then  the  number  of  points  in 
Space  2  is  2  .  In  the  case  of  the  MlAl,  the  corresponding  BRL- 
generated  target  description  is  composed  of  approximately  750 
critical  components.  Since  no  individual  shot  has  a  significant 
likelihood  of  killing  all  components  in  the  target, _the  size  of 
Space  2  for  a  given  Space  1  event  is  far  fewer  than  2'^^  points. 
However  in  just  the  turret-basket  area  of  the  MlAl,  there  are 
some  400  components;  if  only  one-fourth  of  those  components  were 
likely  candidates  for  damage,  there  remain  on^the  order  of  2^°° 
possible  damage  states,  representing  about  10^°  possible  out¬ 
comes.  This  is  too  large  a  number  for  practical  analysis. 

Given  a  particular  damage  state  in  Space  2,  by  definition  a 
set  of  critical  components  is  no  longer  functional.  Thus  there 
may  be  some  reduction  in  the  firepower  or  mobility  function  of 
the  target.  Space  3  represents  an  objective  measure  of  this 
diminution  in  performance.  In  the  case  of  firepower  function, 
characterization  of  Space  3  might  be  in  terms  of  a  reduction  in 
rate  of  fire,  an  increase  in  time  to  acquire  a  target,  or  the 
growth  in  hit  dispersion  of  the  main  gun.  In  the  case  of  mobil¬ 
ity,  Space  3  might  be  represented  by  reduction  of  top  speed, 
reduction  in  acceleration,  or  reduction  in  rough-terrain  crossing 
ability.  Space  3  can  be  thought  of  as  represented  by  objective 
Measures  of  Performance  (MOPs) .  Although  Space  3  is  in  principle 
of  great  interest  to  many  concerned  with  vulnerability  analysis, 
there  is  not  an  implemented  mechanism  for  the  mapping  from  Space 
2  to  Space  3 .  Hence  the  mapping  process  is  represented  with 
dashed  lines  in  Figure  1. 

Although  it  is  theoretically  possible  to  map  each  of  the  2^ 
outcomes  of  Space  2  into  an  objective  MOP  in  Space  3,  this  cannot 
be  practically  accomplished  with  actual  targets.  For  this  rea¬ 
son,  Space  3  will  not  be  further  discussed  here. 

Various  procedures  have  been  used  over  the  years  to  map  Space 
2  directly  into  aggregated  vulnerability  measures  of  effective¬ 
ness  (MOEs)  in  Space  4.  The  traditional  mapping  process  has  made 
use  of  a  set  of  kill  criteria  (which  will  be  discussed  below)  and 
a  Standard  Damage  Assessment  List  (SDAL)  which  maps,  for  mobility 
and  firepower,  the  killed  components  of  Space  2  into  Space  4  MOEs 
that  have  variously  been  called: 

-  Probability  of  Kill  (Pk) 

-  Expected  Loss  of  Function  (LOF) ,  or 

-  Decrement  in  Combat  Utility  (DCU) . 

These  SDAL  mappings  are  estimates,  made  by  groups  of  users,  of 
functional  disturbance  to  vehicle  capability  given  a  killed 
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critical  component.  That  there  are  crippling  problems  with  the 
SDAL  mapping  has  been  documented  elsewhere.  More  recently,  some 
of  these  problems  have  been  overcome  by  defining  target  degraded 
states  explicitly  in  terms  of  component-level  fault  trees.  This 
is  an  attempt  to  divide  up  the  impractically  large  (2")  states  of 
Space  2  into  a  number  of  capability  bins  in  Space  4.  Within  the 
Mobility  function,  for  example,  component  trees  have  been  associ¬ 
ated  with  immobilized,  slight  mobility  loss,  severe  mobility 
loss,  etc.  We  will  have  occasion  to  discuss  the  mapping  to  Space 
(4)  in  more  detail  below. 

3.  Towards  a  Definition  of  Accuracy 

Let  us  return  to  the  accuracy  problem  by  specifying  the  pro¬ 
perties  which  would  have  to  be  true  of  our  vulnerability  analysis 
for  us  to  say  that  our  final  vulnerability  MOE  can  be  stated  with 
arbitrary  precision.  Common  sense  would  seem  to  dictate  that  we 
require  at  least  an  accurate  model  and  accurate  inputs  in  Space 
1;  we  also  require  accurate  calculation  of  the  damage  states  of 
Space  2  and  accurate  vulnerability  estimates  as  output  in  Space 

4 .  For  our  model  to  be  accurate  it  must  have  accurate  algorithms 
for  the  phenomena  treated  and  it  must  also  be  complete  with 
respect  to  the  classes  of  phenomena  we  claim  to  treat.  For  our 
input  to  be  accurate  it  must  be  either  the  result  of  an  accept¬ 
ably  conducted  experiment  or  derived  from  physically  accepted 
first  principles.  For  our  vulnerability  MOEs  to  be  accurate  we 
must  be  convinced  that  they  compare  favorably  with  the  results  of 
experiment. 

I  believe  there  are  two  major  problems  with  the  reasoning  in 
the  previous  paragraph.  First,  the  specific  sense  of  "accuracy" 
intended  for  the  various  parts  of  the  problem  is  not  clear. 
Second,  and  this  has  been  a  frequent  if  mostly  invisible  problem 
over  the  years,  the  task  of  directly  comparing  vulnerability  MOEs 
with  the  results  of  measurements  is  impossible.  We  will  return 
to  "accuracy"  below.  Let  us  first  explain  why  the  Space  4  MOEs 
from  our  models  cannot  be  compared  with  experiment. 

The  Space  4  vulnerability  estimates  —  the  Pks  or  LOF  esti¬ 
mates  —  are  not  physically  observable.  This  is  not  because  the 
estimates  are  (or  are  thought  to  be)  probabilities;  an  experiment 
can  standardly  be  regarded  as  a  single  sample  from  an  underlying 
distribution.  Even  when  the  experiment  is  so  regarded,  it  is  not 


5.  M.W.  Starks,  "New  Foundations  for  Tank  Vulnerability 
Analysis, "  The  Proceedings  of  the  Tenth  Annual  Symposium  on 
Survivability  and  Vulnerability  of  the  American  Defense 
Preparedness  Association,  held  at  the  Naval  Ocean  Systems 
Center,  San  Diego,  CA,  May  10-12,  1988. 
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physically  observable  whether  in  a  given  experiment  the  target  is 
mobility  killed  or  firepower  killed. 

One  reason  why  such  outcomes  are  not  observable  is  that 
whether  a  vehicle  is  killed  or  not  is  a  function  not  only  of  the 
observed  state  of  the  target  (Space  2)  but  also  of  the  functional 
definitions  adopted  for  the  various  forms  of  kill,  and  the 
specifics  of  the  mapping  from  Space  2  to  Space  4.  These  defini¬ 
tions  are  not  empirically  right  or  wrong,  rather  they  are  conven¬ 
tions  which  are  more  or  less  useful  in  serving  the  intended  pur¬ 
poses  of  vulnerability  analysis. 

The  much  reprinted  definitions  of  the  traditional  kill  cri¬ 
teria  are  shown  below; 

-  Mobility:  An  armored  vehicle  experiences  a  mobility  (M) 
kill  if  it  becomes  incapable  of  executing  controlled  movement 
within  a  very  short  time  (0  to  10  minutes)  after  being  hit,  and 
it  is  not  repairable  by  the  crew  on  the  battlefield. 

-  Firepower:  An  armored  vehicle  experiences  a  firepower 
(F)  kill  if  it  becomes  incapable  of  delivering  controlled  fire 
within  a  very  short  time  (0  to  10  minutes)  after  being  hit,  and 
it  is  not  repairable  by  the  crew  on  the  battlefield. 

-  Catastrophic:  An  armored  vehicle  experiences  a  catas¬ 
trophic  (K)  kill  if  it  is  totally  lost  through  .  burning  or 
explosion. 

A  little  reflection  shows  that  these  definitions  are  not  helpful 
for  scientific  vulnerability  analysis.  A  useful  scientific 
definition  requires  the  possibility  of  relatively  unambiguous 
determination,  in  particular  cases,  whether  the  conditions  of  the 
definitions  are  satisfied.  Since  the  definitions  contain  phrases 
such  as  "controlled  fire",  "controlled  movement",  "totally  lost", 
and  "repairable  by  the  crew",  it  is  fairly  clear  that  there  is  a 
great  deal  of  room  (too  much!)  left  for  disagreement  over  cases. 
This  particular  definitional  difficulty  is  avoided  if  degraded 
state  criticality  trees  are  regarded  as  defining  the  various 
classes  of  kill. 

Unfortunately,  the  probability  of  degraded  state  MOEs  are  not 
any  more  experimentally  observable  than  are  the  SDAL  MOEs.  The 
fundamental  reason  for  this  is  that  the  same  probability  of  being 
in  a  degraded  state  can  be  obtained  in  many  different  ways.  An 
example  easily  illustrates  this. 

Figure  2  shows  a  fault  tree  that  defines  the  degraded  state 
for  slightly  reduced  speed.  Suppose  we  fire  1000  live  fire  shots 
under  identical  initial  nditions  with  the  following  outcomes. 
For  223  shots,  none  cf  r.he  components  on  the  reduced  tree  fault 
tree  were  killed,  and  r"  -  777  shots  two  right  support  rollers 
were  killed.  Mapping  this  Space  2  information  to  Space  4  via  the 
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Figure  2. 


Fault  Tree  Defining  the  Degraded  State  Slightly  Reduced 
Mobility 
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fault  tree  we  conclude  that  the  probability  of  the  tank  being  in 
the  slightly  reduced  speed  degraded  state  is  .777. 

Now  suppose  we  wish  to  calculate  vulnerability  MOEs  in  a 
model  for  the  initial  and  boundary  conditions  of  the  Live  Fire 
Test.  If  we  run  1000  monte  carlo  cases  in  our  model  we  might 
obtain  the  Space  2  result  that  for  223  cases  no  critical  com¬ 
ponent  was  damaged  and  for  777  cases  left  road  wheel  #7  was 
killed.  Our  mapping  to  Space  4  will  once  again  yield  the  proba¬ 
bility  of  degraded  state  result  of  .777.  Our  comparison  of 
experiment  with  theory  gives  identical  kill  probabilities  even 
though  there  are  no  killed  components  in  common. 

In  light  of  the  simple  example,  it  becomes  clear  what  can  and 
cannot  be  correctly  inferred  from  MOE  agreement  or  disagreement 
in  Space  4  concerning  damage  state  agreement  or  disagreement  in 
Space  2.  As  the  example  shows,  we  cannot  infer  agreement  in  cal¬ 
culated  damage  states  (Space  2)  from  agreement  of  MOEs  in  Space 
4.  Even  if  we  had  complete  agreement  in  MOEs  between  experiment 
and  simulation,  this  leaves  open  the  possibility  that  the  actual 
and  predicted  Space  2  damage  states  are  completely  disjoint. 

Most  people  will  accept  as  a  necessary  condition  on  model 
validity  that  critical  model  calculations  be  acceptably  accurate; 
it  is  not  an  adequate  procedure  to  compare  Space  4  MOEs  and 
declare  that  our  model  is  accurate  if  they  agree.  Surprisingly, 
some  workers  have  come  perilously  close  to  doing  exactly  that. 

Pollard  et  al  , for  example,  use  correlation  analysis  to 
relate  Space  4  Live  Fire  Test  outcomes  to  calculated  MOEs.  After 
presentation  of  the  correlation  technique  employed  and  display  of 
the  comparative  results,  they  say: 

"Thus,  while  additional  effort  is  required  on  the  models 
to  achieve  good  correlation  for  all  classes  of  weapons 
and  targets,  the  use  of  the  expected  value  model  predict¬ 
ions  as  average  characterizations  of  vehicle  vulnerability 
for  wargaming  purposes  appears  reasonable." 

This  is  a  very  weak  conclusion  and  it  is  heavily  qualified. 
None-the-less ,  based  on  the  argument  of  the  last  few  paragraphs 
it  is  simply  false  that  high  correlations  of  the  s  ort  indicated 
give  any  warrant  that  our  models  and  their  output  MOEs  are 


6.  R.G.  Pollard,  G.L.  Holloway,  D.C.  Bely,  F.  Tyler  Brown  and 
J.C  Kisko,  "An  Examination  of  Vulnerability  Predictions  in 
Light  of  Live  Fire  Testing  of  Light  Combat  Vehicles,"  The 
17*"  TWG/AOR  Quadrapartite  Working  Group,  for  AOR,  Sydney 
Australia,  December  1987. 
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"reasonable".  As  the  simple  example  showed,  even  a  correlation 
coefficient  of  one  is  insufficient  to  guarantee  that  our  model 
agrees  with  experience. 

We  have  seen  so  far  that  Space  4  is  an  unacceptable  place  to 
look  if  the  accuracy  of  our  models  is  to  be  clarified.  It  can 
too  easily  show  apparent  agreement  between  Pks  while  masking 
serious  defects  in  the  details.  The  only  useful  purpose  that 
Space  4  comparisons  have  for  the  accuracy  issue  is  a  negative 
one.  That  is,  if  there  is  disagreement  in  Space  4  MOEs  between 
simulation  and  experiments,  we  can  infer  that  we  have  a  disagree¬ 
ment  in  the  Space  2  damage  states.  The  natural  suggestion 
arises:  why  not  look  directly  at  Space  2  for  measures  of 
model/experiment  agreement  in  the  first  place?  Before  exploring 
this  suggestion  in  more  detail,  it  is  worth  pointing  out  that 
when  we  change  our  domain  of  interest  from  Space  4  to  Space  2 
there  is  no  loss  of  information.  We  know  that  Space  2  agreement 
between  model-predicted  and  experimental  damage  states  strictly 
implies  Space  4  agreement. 

It  is  a  natural  and  understandable  chain  of  reasoning  that 
since  the  VLD's  primary  output  is  probability  of  kill  (Pk) ,  and 
since  we  have  computer  codes  calculating  Pks,  then  model  accuracy 
and  validation  should  have  to  do  with  Pks.  Unfortunately,  the 
reasoning,  though  understandable,  is  unsound. 

A  final  heuristic  point  may  further  illuminate  why  we  should 
look  to  the  damage  states  rather  than  to  the  final  vulnerability 
MOEs  if  we  wish  to  clarify  the  issue  of  vulnerability  model  accu¬ 
racy.  In  principle,  the  modeling  work  for  Spaces  1-3  can  proceed 
according  to  acceptable  scientific  standards.  The  terms  that  we 
use  to  predict  component  loss  of  function  such  as  mass,  velocity, 
and  drag  are  certainly  familiar  from  the  lexicon  of  theoretical 
physics.  Thus,  even  though  there  is  no  first  principles  vulnera¬ 
bility  science  to  use  in  simulation  modeling,  there  is  a  rich 
variety  of  empirical  generalizations  available  covering  many  cen¬ 
tral  vulnerability  phenomena.  This  is  not  to  say  that  we  have  an 
empirical  law  available  for  each  vulnerability  phenomenon  we  are 
interested  in.  However,  we  do  have  standard  modes  of  scientific 
experiment,  inference,  and  argument  available  to  help  us  make 
further  progress  and  to  help  adjudicate  disputes. 

The  scientific  standards  which  can  be  brought  to  bear  when 
discussing  Space  2  damage  states  are  no  longer  available  once 
these  damage  states  have  been  mapped  to  Space  4  MOEs.  As  was 
discussed  above,  these  mappings  either  are  or  require  use  of 
definitions  which  from  a  scientific  point  of  view  are  essentially 
arbitrary.  The  evaluative  criteria  for  the  usefulness  of  a  par¬ 
ticular  form  of  Space  4  MOE  are  not  those  of  empirical  science, 
rather  they  are  pragmatic.  Does  a  given  MOE  illuminate  the  vul¬ 
nerability  issue  or  issues  that  require  clarification?  As  a 
practical  matter,  the  answer  to  this  question  will  depend  on 
whether  the  interlocutor  is  a  commander,  a  representative  of  the 
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user,  a  vehicle  designer,  a  weapon  designer,  a  war-gamer,  a 
senior  decision-naker ,  a  tactician,  or  is  in  some  other  role. 
Not  surprisingly,  an  MOE  may  be  regarded  as  useful  by  some  and 
useless  by  others.  There  are  sometimes  heated  arguments. 
Because  the  Space  4  MOEs  purport  to  assess  the  capability  of  the 
target  in  warfare,  adjudication  of  MOE  goodness  necessarily 
requires  postulations  and  speculations  concerning  heretofore 
unfought  wars.  This  is  a  long  way  from  masses  and  velocities. 

I  do  not  wish  to  be  understood  as  arguing  that  discussions 
about  Space  4  MOEs  are  useless.  BRL  must  keep  an  oar  in  the 
water  here  to  insure  that  our  MOEs  are  as  useful  as  they  can  be 
to  meet  the  diverse  needs  of  our  many  types  of  customer.  What  I 
am  arguing  is  that  we  should  distinguish  as  clearly  as  possible 
between  the  scientific  criteria  associated  with  our  Space  2 
damage-state  work,  where  accuracy  can  be  defined,  and  our  Space  4 
MOE  work  where  accuracy  cannot  be  defined.  We  could  say  that  in 
Space  2  we  aspire  to  vulnerability  science,  but  in  Space  4  we  can 
do  no  better  than  pragmatically  acceptable  analysis. 

4.  ACCURATE  VULNERABILITY  PREDICTIONS 

We  have  settled  on  the  Space  2  damage  states  as  the  appropri¬ 
ate  place  for  comparisons  between  vulnerability  models  and  vul¬ 
nerability  experiments.  If  we  can  ’'validate”  our  model  in  Space 
2  we  can  guarantee  agreement  in  Space  4. 

As  noted  above,  the  BRL  target  description  for  the  Abrams 
tank  is  composed  of  roughly  750  critical  components.  Under  our 
assumption  that  for  a  test  event  we  can  regard  each  component  as 
dead  or  alive  (Appendix  A) ,  we  can  give  a  complete  scientific 
account  of  the  outcome  of  a  Live  Fire  shot  by  specifying  an  evi¬ 
dence  vector  E  whose  components  c...c  take  the  values  1  (for 
alive)  or  0  (for  dead) .  For  each  monte-cSrlo  iteration  of  our 
vulnerability  model  we  generate  a  hypothesis  vector  H  whose 
identically  numbered  components  also  take  the  values  1  and  0. 

Suppose  we  have  a  statistically  significant  number  of  vulner¬ 
ability  experiments  with  identical  initial  conditions  and  have 
characterized  the  outcome  of  each  with  an  appropriate  E  vector. 
Suppose  further  that  we  also  have  an  appropriate  series  of  H  vec¬ 
tors  as  output  from  a  monte-carlo  model  simulating  those  same 


7.  Sometimes  these  MOE  arguments  are  made  by  scientists.  This 
results  in  the  spectacle  of  "fog  of  war”  anecdotal  musings 
posed  as  if  they  were  scientific  arguments  for  or  against  a 
particular  MOE.  Attention  to  the  difference  between  scien¬ 
tific  and  pragmatic  evaluative  criteria  would  mitigate  some 
of  the  worst  features  of  these  arguments. 
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initial  conditions.  For  such  a  case,  there  are  well  established 
statistical  methods  for  answering  the  question  of  whether  we  can 
be  confident  that  the  H  and  E  distributions  are  the  same.  The 
degree  of  confidence  with  which  we  can  claim  that  they  are  the 
same  characterizes  the  accuracy  of  our  model  for  the  initial  con¬ 
ditions  under  study. 

Unfortunately,  large  empirical  sample  sizes  for  identical 
initial  conditions  are  prohibitively  costly  in  system  vulnerabil¬ 
ity  testing.  For  problems  of  practical  interest  we  will  often 
have  a  single  E  vector  to  compare  with  multiple  H  vectors.  What 
can  we  learn  about  the  accuracy  of  our  models  when  faced  with 
this  type  of  comparison?  For  a  single  shot,  the  emerging  answer 
seems  to  be  "very  little”,  or  perhaps  even  "nothing”. 

It  deserves  repeated  emphasis,  however,  that  our  success  or 
failure  in  articulating  the  extent  to  which  our  models  can  accu¬ 
rately  predict  experimental  outcomes  is  squarely  predicated  on 
consideration  of  the  relevant  H  and  E  vectors  only.  Considera¬ 
tion  of  Space  4  Pks  contributes  no  additional  cognitive  content 
once  we  have  assessed  the  situation  in  Space  2 . 

5.  A  CONSEQUENCE  FOR  COMPARTMENT  MODELS 

In  the  sense  of  the  phrase  "accurate  prediction”  developed 
here,  only  component  level  models  can  even  have  the  possibility 
of  making  accurate  predictions.  So  called  compartment  models, 
which  map  directly  from  Space  1  to  an  MOE  in  Space  4  are  neither 
accurate  nor  inaccurate.  The  best  that  can  be  hoped  for  with 
this  class  of  model  is  that  its  Space  4  MOES  agree  with  a 
component-level  model's  Space  4  MOES,  where  the  component-level 
model  has  been  independently  judged  to  be  accurate  in  the  Space  2 
sense.  Unfortunately,  for  cases  where  this  much  information  is 
available,  the  compartment  model  is  not  really  needed.  It  is 
when  we  don't  have  component-level  information  that  the  compart¬ 
ment  model  is  most  used,  and  it  is  precisely  for  those  cases  that 
we  cannot  assess  its  accuracy.  This  forces  a  conclusion  which  I 
believe  is  true:  we  do  not  know  how  accurate  most  compartment- 
level  predictions  are. 

6.  SUMMARY  OF  MAIN  ARGUMENT 

System  probability  of  kill  is  not  a  field  observable;  more¬ 
over  identical  probability  of  kill  values  can  issue  from  com¬ 
pletely  different  combinations  of  killed  components.  Thus,  pro¬ 
bability  of  kill  is  the  wrong  level  at  which  to  compare  vulnera¬ 
bility  theory  with  experiment.  A  better  level  is  the  damage 
state;  this  is  specified  by  determining  (after  an  actual  or  simu¬ 
lated  shot)  which  target  components  are  killed  and  which  are 
alive. 
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APPENDIX  A  -  Components:  Dead  or  Alive 


Our  earlier  discussion  of  Space  2  damage  states  required  the 
simplifying  assumption  that  for  given  v;eapon  target  encounter 
conditions  each  component  would  either  be  killed  or  not.  Thus, 
the  kills  for  the  components  would  be  ones  or  zeroes.  Unfor¬ 
tunately,  the  received  wisdom  on  vulnerability  analysis  does  not 
always  divide  things  so  neatly. 

One  problem  is  that  it  is  not  possible  to  give  a  general 
account  of  when  we  should  declare  an  arbitrarily  chosen  component 
killed.  We  might  say  that  a  component  is  dead  if  it  can't  ful¬ 
fill  its  function  in  the  tank,  but  this  leaves  plenty  of  room  for 
disagreement  about  cases.  I  do  not  believe  that  this  difficulty 
can  be  avoided  in  any  practically  implementable  program  for  vul¬ 
nerability  analysis.  We  must  just  face  the  fact  that  we  must 
think  through,  for  each  component,  what  its  function  is  and  what 
level  of  damage  would  be  required  to  prevent  that  function  from 
occurring . 

What  about  the  issue  of  component  probability  of  kill  esti¬ 
mates  which  have  historically  been  regarded  as  other  than  binary? 
There  are  a  few  different  kinds  of  cases  that  seem  most  prob¬ 
lematic.  For  example,  a  copper  fluid  line  may  be  bent  by  a  frag¬ 
ment  impact  such  that  fluid  flow  is  restricted  but  not  completely 
stopped.  Our  fluid  line  is  neither  dead  nor  alive.  Rather,  it 
has  a  certain  fraction  of  its  original  capability.  Therefore, 
the  argument  goes,  any  theoretical  structure  for  vulnerability 
analysis  which  makes  the  assumption  that  each  component  is  dead 
or  alive  is  intrinsically  flawed. 

First,  it  must  be  granted  that  it  is  at  least  sensible  to 
discuss  the  fractional  capability  of  at  least  some  types  of  com¬ 
ponents.  If  our  copper  line  previously  had  a  capability  of  10 
liters/minute  and  now  can  carry  only  5  liters/minute,  then  it 
certainly  seems  sensible  to  assert  that  the  line  has  .5  frac¬ 
tional  capability.  Of  course  there  is  still  a  question  about  the 
consequences  of  a  line  with  .5  fractional  capability  on  vehicle 
operation.  But  this  should  be  a  question  we  can  satisfactorily 
answer,  at  least  in  theory. 

There  are  points  of  both  similarity  and  difference  between 
the  case  of  the  fluid  line  and  the  case  of  the  overall  tank. 
Talk  about  a  tank's  fractional  capability  (combat  utility)  is 
vague  to  the  edge  of  being  meaningless.  On  the  other  hand,  it 
makes  sense  to  consider  the  fractional  capability  of  a  fuel  line. 
However,  for  neither  case  does  it  make  sense  to 
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talk  about  fractional  capability  when  it  is  convenient  to  do  so, 
and  to  regard  the  quantities  as  probabilities  when  it  is  con¬ 
venient.  If  we  were  to  allow  treatment  of  a  fluid  line  with  .5 
capability  as  if  it  were  a  fluid  line  with  a  .5  probability  of 
destruction,  we  v;ould  be  making  a  serious  mathematical  error. 

Fortunately,  I  believe  that  the  fuel  line  can  be  handled 
without  making  such  an  error;  these  difficulties  can  be  success¬ 
fully  accommodated  within  the  framework  I  have  been  advocating  in 
this  paper.  How  should  we  handle  our  critical  fluid  line  with  .5 
fractional  capability?  The  short  answer  is:  it  depends  on  what 
it  does  to  the  operation  of  the  tank.  Suppose  we  use  pliers  and 
crimp  the  fluid  line  such  that  it  has  .9,  .8,  .7  ...  0.0  of  its 
original  capacity.  After  each  crimping  we  try  to  use  the  tank 
for  the  type  of  task  which  the  damage  criterion  is  designed  to 
illuminate.  At  some  point  (possibly  only  when  we  get  to  0  frac¬ 
tional  capability)  we  will  no  longer  be  able  to  execute  that  type 
of  task.  That  is  the  point  at  which  we  declare  our  fluid  line  to 
be  dead.  What  we  are  essentially  doing  is  defining  a  threshold 
of  fractional  capability  beyond  which  the  component  is  declared 
legally  dead. 

Somebody  might  argue  that  the  questions  of  whether  a  fragment 
of  a  given  mass  and  velocity  crimps  the  fluid  line  to  exactly  the 
kill-threshold  level  and  whether  there  even  exists  a  precise 
threshold  level  that  causes  task  failure  are  best  answered  sto¬ 
chastically.  And  it  must  be  admitted  that  our  ignorance  of  the 
precise  initial  and  boundary  conditions  of  the  vulnerability 
problem  are  such  that  stochastic  methods  are  appealing;  this  is 
why  such  models  have  recently  been  developed  in  VLD.  However,  I 
cannot  see  that  component  kill  information  is  inherently  "more 
stochastic"  than  other  required  point  burst  inputs,  for  example, 
penetration.  In  a  Monte  Carlo  model  we  should  certainly  form 
distributions  of  every  input  variable  we  can,  including  component 
PKHs  if  we  have  the  information  to  develop  the  distribution,  but 
for  expected  value  modeling  I  do  not  see  any  harm  in  regarding 
component  death  as  a  binary  matter. 
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